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ABSTRACT : Ecological Impact Assessment (EIA) is instrumental in guarding biodiversity from ill-planned development projects. 
Much of EIA involves field surveys of different taxonomic groups and their habitats, but little information is available on what 
factors influence the performance of these surveys. In this study, we used a recently developed evaluation checklist to understand 
the state of biodiversity surveys. We asked two questions: 1) Have the installment of laws and guidelines improved survey 
performance? 2) Does surveyor background affect the performance of surveys? The EIA reports in Taiwan were our study subjects. 
The results show that the majority of ecological impact assessments are performed poorly, although the average quality of reports 
has increased significantly over time, with legislation playing a key role in improving the quality of surveys. We also found that the 
background of surveyors had significant effects on survey reports. The effects need to he interpreted with care. We propose that 
training and certification be required for personnel performing surveys. 


KEY WORDS: Biodiversity, ecological survey, effectiveness 
system. 

INTRODUCTION 

Most development projects inevitably cause the loss 
of natural habitats. Habitat loss is often accompanied by 
habitat isolation and degradation (de Chazal and 
Rounsevell, 2009; Lindenmayer, 2009; Swift and 
Hannon, 2010). Together, they can lead to irreversible 
negative effects on local biodiversity (Slootweg and 
Kolhoff, 2003). The ecological impact assessment (EIA, 
hereafter) is a key component in the Environmental 
Impact Assessment. The former examines the potential 
ecological consequences, particularly the loss of 
biodiversity, of proposed development projects (Wood, 
2003a; Chadwick et ah, 2005; Mandelik et ah, 2005; 
Soderman, 2006; Pinho et ah, 2007; Samarakoon and 
Rowan, 2008; Morris and Therivel, 2009). The EIA 
generates scientific data which public administrators 
can use, along with other information, to make rational 
decisions regarding development policies (Downs, 
2008; Gasparatos et ah, 2008; Hacking and Guthrie, 
2008; Song and Glasson, 2009). It is meant to be a 
constructive tool promoting sustainable developments. 

However, in some countries, the integrity of 
Environmental Impact Assessment has been called into 
question by conservation groups and the general public. 
For example, Chen (2002) phone-interviewed residents 
in Taiwan about their perceptions of environmental 
impact assessment. He found that 50% of the public 
thought the assessment process was often corrupted. 


, Environmental Impact Assessment, EIA reports, evaluation 

and the assessment results were not trust-worthy. 
Furthermore, Chen (2000) suggested that the conflicts 
between conservation groups and developers have 
become a source of societal instability. In order to 
preserve the integrity of Environmental Impact 
Assessment, it is important to evaluate its performance 
and to understand what factors influence its quality. 
However, little information is available. 

The Taiwanese government began exercising 
Environmental Impact Assessment on the majority of 
public development projects in the late 1970s, and 
passed the Environmental Impact Evaluation Act, 
which mandated Environmental Impact Assessment, in 
1994 (Liou and Yu, 2004). EIA, a key component in the 
Environmental Impact Assessment, involves field 
surveys of various taxonomic groups and their habitats. 

It requires adequate survey methodologies to provide 
accurate, systematic, and quantified predictions of 
impacts on biodiversity (Treweek, 1996; Thompson, 
2007; Thompson et al., 2007; Geneletti, 2008). In 
Taiwan, the guidelines for animal and vegetation 
surveys were not established until 2003 (Chen et ah, 
2009), providing a foundation for better EIA. In order 
to safe-guard the effectiveness of EIA in Taiwan, we 
aimed to objectively evaluate the quality of EIA. To do 
so, we developed a checklist for assessing the quality of 
EIA reports (Lee, 2009). Since EIA reports are a major 
product of the EIA process, and high quality reports 
can contribute to better decision-making, we believe 
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that the evaluation of EIA reports is instrumental in 
measuring the effectiveness of the EIA process (Wood, 
2003b, 2008; Ramjeawon and Beedassy, 2004; 
Demidova and Cherp, 2005; Hartley and Wood, 2005; 
Pinho et ah, 2007; Geneletti, 2008). The checklist was 
developed using the Delphi expert system with the help 
from 17 experts that had extensive experiences in 
performing and/or reviewing EIA (see below). While 
developing the checklist, we also asked the experts to 
rank several important factors that could influence the 
quality of ecological surveys. The experts consistently 
placed the credential of surveyors the topmost factors 
affecting EIA quality (Lee, 2009). 

In this study, we used the checklist to evaluate the 
quality of EIA reports bulletined between 1988 and 
2006 by the Taiwanese government. With an aim to 
provide recommendations for improvements in EIA 
performance, we addressed two questions. First, 
because the government institutes or amends 
environmental impact assessment laws over time, we 
asked: Has the installment of laws and guidelines 
improved EIA performance? Second, because the 
experts consistently consider the credential of surveyor 
paramount important, we asked: Does surveyor 
background affect the performance of EIA? The EIA 
reports of development projects in Taiwan during 
1988-2006 were our study subjects. 

MATERIALS AND METHODS 

EIA reports 

Between 1988 and 2006, 1067 Environmental 
Impact Assessment reports were bulletined by the 
Environmental Protection Administration of the 
Taiwanese government. We divided all reports into 
three chronological groups: 1988-1994, 1995-2003, 
2004-2006 (Group I, II, and III, respectively) based on 
two major legislative events: the installation of the 
Environmental Impact Evaluation Act in 1994 and the 
release of the Animal Ecology Assessment Technique 
Guidelines and Plant Ecology Assessment Technique 
Guidelines by the government in 2003. We randomly 
sampled approximately one half of the reports, 524 
reports overall with 150, 253, and 121 reports from 
chronological group I, II, and III, respectively. The 
ecological impact assessment sections of those reports 
were evaluated with a comprehensive checklist 
(available upon request, Lee, 2009). The checklist was 
established by the Delphi expert system (Hwang, 2000; 
Yasamis, 2006). The experts who participated in 
producing the checklist were 17 established Taiwanese 
ecologists with various taxonomic expertise and 
experience in performing and/or reviewing EIA (Lee, 
2009). 


In Taiwan, depending on the type of development 
project, ecological assessment for terrestrial ecosystems, 
aquatic ecosystems, or both is required. Hence, we 
further classified the 524 EIA reports into three 
categories: Land, Water, and Land-Water, respectively. 
Sample sizes of the three categories in the three 
chronological groups were 62, 16, 73 in group I, 94, 6, 
153 in group II, and 52, 1, 68 in group III. Finally, we 
divided the EIA reports into 5 clusters according to the 
surveyors’ academic background (Table 1): (A) 

background not specified; (B) the surveyors had a BS or 
higher degree in civil engineering, environmental 
engineering, or similar field; (C) a BS or higher degree 
in leisure study, landscape architecture, or similar field; 
(D) a BS or higher degree in oceanography, 
horticulture, animal science, or similar field; (E) a BS or 
higher degree in biology, life science, zoology, botany, 
or ecology (Table 1). The clustering criteria were based 
on the number of ecology-related courses needed to 
fulfill the requirements for a degree in that professional 
field. 

Evaluation system and process 

The EIA evaluation checklist we used contained 
five sections: description of ecological environment 
(area, topography, etc.), schedule of survey (month, 
intensity, etc.), survey technique (sampling tools used, 
etc.), presentation of results (statistical presentation, 
etc.), and presentation of suggestions (discussing 
relevant results, etc.). The details of the checklist are 
available upon request (Lee, 2009). The five sections 
were scored against 7, 9, 56-127, 29-87, and 1-3 
scoring items, respectively, depending on the types of 
ecosystems assessed. In Taiwan, the EIA for terrestrial 
ecosystems is required to provide information on plants, 
mammals, birds, reptiles, amphibians, and butterflies; 
while EIA for aquatic ecosystems is required to provide 
information on fish, aquatic insects, and planktons. 
Thus, the actual number of scoring items differed 
among the three ecosystem categories. The scoring 
checklist was a series of binary yes or no questions. 
Each scoring item was given 1 or 0 for yes or no, 
respectively. For example, if a map indicated the survey 
site was given in the report, we considered it to have 
fulfilled the scoring item “survey area description”, and 
the item would be scored 1. We did not differentiate the 
degree of detail given within each scoring item. Hence, 
reports receiving high scores indicate a high degree of 
fulfillment of survey items deemed important as 
suggested by the 17 experts. In all, the highest possible 
scores for Land, Water, and Land-Water reports were 
177, 102, and 244, respectively. A detailed description 
of scoring technique was given in Lee (2009). We 
considered reports that fulfilled 60% of the required 
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Table 1. The number of EIA reports sampled from three 
ecosystem categories and five surveyor clusters: (A) 
surveyors’ backgrounds not specified; (B) surveyors had a 
BS or higher degree in civil engineering, environmental 
engineering, or similar field; (C) surveyors had a BS or 
higher degree in leisure study, landscape architecture, or 
similar field; (D) surveyors had a BS or higher degree in 
oceanography, horticulture, animal science, or similar field; 
(E) surveyors had a BS or higher degree in biology, life 
science, zoology, botany, or ecology. 


Ecosystem 


Surveyor 

Land 

Water 

Land-Water 

Total 

A 

85 

19 

82 

186 

B 

30 

2 

32 

64 

C 

27 

0 

43 

69 

D 

13 

0 

32 

45 

E 

52 

2 

105 

160 

Total 

207 

23 

294 

524 


Note: Three ecosystem categories: Land—EIA was required for 
terrestrial ecosystems only, Water—EIA was required for aquatic 
ecosystems only, Land-Water—EIA was required for both 
ecosystems. 

items good reports, and the rest poor. 

We took several measures to reduce biases. First, 
one person (YHL) performed all the scoring. Second, 
the scorer practiced scoring with the checklist on 200 
randomly selected reports before formal scoring started. 
Third, the scoring was performed in a random order 
regarding chronological groups, ecosystem categories, 
and surveyor clusters. Finally, the identities of the 
surveyors were blocked from the scorer. 

Statistical analyses 

We used Lilliefors’ tests (a one-sample test that 
compares a sample with a normal distribution) to test 
for normality of the data set sorted in different ways: 
chronological groups, ecosystem categories, and 
surveyor clusters. For data sets that passed the 
normality tests, we used Tukey’s honestly significant 
difference tests, (a parametric multiple 
comparison procedure that compares all possible pairs 
of means) to compare between treatments, and used 
Scheffe’s tests (a parametric test that compare pairs of 
samples) for post hoc comparisons. For data sets that 
failed the normality tests, we used Kmskal-Wallis 
one-way ANOVAs (a nonparametric equivalent of 
ANOVA), followed by Kolmogorov-Smimov tests (a 
nonparametric test that compare pairs of samples) for 
post hoc comparisons. We adjusted significance levels 
in all post hoc tests to account for multiple 
comparisons. Because sample size was too small in the 
Water reports (n=23) for meaningful analysis, we did 


not look for the effect of surveyor background on them. 

RESULTS 

The highest possible scores for Land, Water, and 
Land-Water reports were 177, 102, and 244, 

respectively. The scores received by Land reports 
ranged from 0 to 153 and averaged 64.0±3.3 
(meanilse). Twenty six percent of the reports received 
>60% of the possible score. The scores received by 
Water reports ranged from 17 to 98 and averaged 
55.4±4.9. Forty three percent of the reports received 
>60% of the possible score. The scores received by 
Land-Water reports ranged from 0 to 214 and averaged 
113.6±3.3. Thirty two percent of the reports received 
>60% of the possible score. The performance of reports 
in terms of the percent of the maximum score received 
differed significantly among the three ecosystem 
categories (Kruskal-Wallis test, Ftc=12.8 > X^o.os, 2=6.0, 
P < 0.002). The Land reports performed significantly 
worse than the Water and Land-Water reports. Overall, 
approximately 1/3 of EIA reports received >60% of the 
maximum possible score over the past 30 years (Fig. 1). 

The EIA reports performed well in description of 
ecological environment (section 1) and presentation of 
suggestions (section 5), with 80% of the reports scoring 
satisfactorily (fulfilling > 60% of the survey items in 
that category) in those areas. The EIA reports did 
poorly in the other three areas (schedule of survey, 
survey technique, and presentation of results), with 
barely 40% of the reports scoring satisfactorily in those 
areas. 

In general, the qualities of reports improved over 
time. The mean scores of both Land and Land-Water 
reports increased significantly from chronological group 
I to III (Kruskal-Wallis tests, Hc=33.5 > x^o.os, 2=6.0, P < 
0.0001 for Land reports; Hc=177.4 > x^o. 05 ,2=6.0, P < 
0.005, for Land-Water reports; Table 2). The sample 
size of Water reports (n=23) was too small to perform 
meaningful statistical tests. 

The background of surveyors had a significant 
effect on the quality of reports. Generally, surveyors in 
cluster A or cluster B performed worse than surveyors 
in the other clusters (Kruskal-Wallis tests, FIc=85.3 > 
X^o.o5,4=9.5, P < 0.0001 for Land reports; Hc=66.0 > 
X^o.o5,4=9.5, P < 0.0001, for Land-Water reports; Table 
3). Among the Land reports, 58% of the C, D and E 
clusters received > 60% of the maximum possible 
score; among the Land-Water reports, 53% of the C, D 
and E clusters received > 60% of the maximum possible 
score (Fig. 1). 

DISCUSSION 

The EIA component of Environmental Impact 
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A 



B 


•c 

o 

a. 


(D 

E 


0 


Evaluation Scores 



0 1-20 21-40 41-60 61-80 81-102 


C 


Evaluation Scores 



0 1^8 49-96 97-144 145-192 193-244 

Evaluation Scores 

Fig. 1. The evaluation scores (qualities) of EIA reports for the 
three ecosystem categories: (A) Land— EIA was required for 
terrestrial ecosystems only, (B) Water— EIA was required for 
aquatic ecosystems only, and (C) Land-Water— EIA was 
required for both ecosystems. The numbers above the bars 
give sample sizes. The dashed lines indicate the cut-off of 
receiving > 60% of possible highest scores. The highest 
possible scores for Land, Water, and Land-Water reports 
were 177, 102, and 244, respectively. 


Assessment is erueial in preventing ill-planned 
development projeets from degrading eeosystems and 
deereasing biodiversity. It is a eritieally important 
proeess in developing nations, ineluding Taiwan, where 
eeonomie growth is aeeompanied by swift ehanges in 
land use patterns. However, our results indieate that the 
EIAs in Taiwan fall short in aehieving their goal as 


Table 2. The mean (±1se) scores received by Land and 
Land-Water reports during the three time periods, I 
(1988-1994), M (1995-2003), & III (2004-2006). Numbers in 
parentheses give sample sizes. Different lower case letters 
in each column denote a significant difference between time 
periods within an ecosystem category based on post hoc 
comparisons using Kolmogorov-Smirnov two-sample tests. 
The highest possible scores for Land and Land-Water 
reports were 177, and 244, respectively. 


Period 

Land 

Land-Water 

I 

M2.4±3.69 (61) 

‘>95.6±6.29 (73) 

n 

”73.^4.78 (95) 

”113.8±16.46(151) 

m 

"9L2T5.48 (52) 

” 129.2±5.58 (68) 


Note: see Table 1 for the explanation of ecosystem categories. 


judged from the quality of the EIA reports. Only 1/3 of 
EIA reports reeeived >60% of the maximum possible 
seore during 1988-2006 (Fig. 1). Although this result is 
both frustrating and alarming, we were glad to find that 
the seores of EIA reports have improved signifieantly 
over the three time periods. The trend of improvement 
eoneurred with eonelusions from similar studies 
(Vanderhaegen and Muro, 2005; Pinho et al., 2007), and 
strongly suggested the positive effeets of legislation on 
the quality of EIA reports. We hope this result would 
lead to more innovative new laws that will guide EIA 
toward even better performanee in the future. 

However, legislative efforts alone are not enough. 
The Taiwanese government released both the Animal 
Eeology Assessment Teehnique Guidelines and the Plant 
Eeology Assessment Teehnique Guidelines in 2003, 
requiring surveyors to follow the guidelines during EIA. 
We still found a large pereentage of EIA reports perform 
poorly after 2003. This indieates an urgent need for a 
meehanism to ensure the quality of EIA reports in 
Taiwan. The Environmental Proteetion Administration 
and other related governmental agencies ought to confer 
on a way to monitor the quality of EIA. We believe an 
evaluation process similar to the one used in this study 
provides a mean to safe-guard EIA performance. 

The evaluation also revealed directions for 
improvement. For example, we found that the majority 
of EIA reports did poorly in the presentation of schedule 
of survey, survey technique, and survey results. One may 
argue that poor presentation does not necessarily mean 
lousy assessment process. However, EIA reports ought 
to provide sound scientific data, and probably the only 
such data, to public administrators that could be 
consulted to make decisions regarding development 
policies. It is critical the presentation is rigorous, 
faithfully reflecting both the assessment process and the 
condition of ecosystem being assessed. Schedule of 
survey, survey technique, and presentation of results 
provide important information for independent 
assessment of potential impacts, thus, should be carefully 
documented in the EIA reports. 
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The checklist we produced was established by 
consulting the guidelines as well as 17 established 
Taiwanese ecologists using the Delphi expert system. 
Hence, the checklist was composed of survey items 
deemed important as suggested by experts (Lee, 2009). 
It is worth noting that the tool we used in the evaluation 
process was only a checklist. We did not differentiate 
the degree of detail given for each scoring item on the 
checklist. There was a wide range of detail which was 
not considered by the checklist (Lee, 2009). For 
example, the EIA reports tended to focus on protected 
species, and confined themselves to on-site changes, 
and lacked assessment at the ecosystem level and at the 
spatial and temporal scales of ecological processes. 
These are critical problems that we are unable to detect 
with our checklist. Thus, we should not be satisfied with 
the checklist as it is. 

We found the background of surveyors had a 
significant effect on the quality of EIA (Table 3). Such 
an effect should be interpreted with care. Knowing the 
details of many EIAs, we do not think our results 
indicate the background of surveyors affect the 
competency of performing biodiversity survey. 
Alternatively, it is possible that surveyor background 
does not have a direct effect, but is correlated with some 
other factors that directly affect EIA report quality. For 
example, we found that in some cases, the responsible 
personnel surveying a taxonomic group (e.g., birds) 
were the same persons surveying some other taxonomic 
group (e.g., plants). Sometimes, the responsible 
personnel surveying biodiversity (e.g., birds) were the 
same persons performing some other assessments (e.g., 
geology). That is, some projects failed to recruit 
appropriate man-power for biodiversity survey, and 
could result in poor EIA reports. The lack of 
appropriate man-power may be a direct consequence of 
poor funding. Other than man-power and limited 
funding, many other factors such as tight deadlines and 
inadequate report review could also affect EIA report 
quality. Unfortunately, we were unable to analyze how 
these factors just mentioned affect EIA report quality 
because the information is lacking or incomplete in 
most reports. They are not within the scope of this 
study. Future researches looking into the effects of these 
and other factors related to EIA are badly needed. Lee 
(2009) touched on some of the issues in three case 
studies. 

Nevertheless, the 17 experts we consulted 
consistently named the credential of surveyors as one of 
the chief factor affecting EIA quality (Lee, 2009). In 
light of our finding of surveyor background effect, we 
agree that it is necessary to install a certification process 
for all personnel who plan to perform biodiversity 
survey for EIA. A certain level of training in ecology 
can ensure better quality in EIA. In the past few years. 


Table 3. The mean (±1se) scores of Land and Land-Water 
reports prepared by surveyors with different background. 
Numbers in parentheses give sample sizes. Different lower 
case letters in each column denote a significant difference 
between surveyor clusters within an ecosystem category 
based on post hoc comparisons using 
Kolmogorov-Smirnov two-sample tests. The highest 
possible scores for Land, and Land-Water reports were 177, 
and 244, respectively. 


Surveyor 

Cluster 

Land 

Land-Water 

A 

!134.14±4.23 (85) 

‘>80.85±11.55 (82) 

B 

b 46.94±6.40 (30) 

"76.92±5.49 (32) 

C 

"82.U7.33 (27) 

” 119.71±6.36 (43) 

D 

" 105±9.21 (13) 

126.9±6.66 (32) 

E 

" 102.8±5.05 (52) 

" 144.37±3.95 (105) 


Note: see Table 1 for the explanation of ecosystem categories; see 
Table 2 for the explanation of surveyor clusters. 


we have seen progress on this front. For example, the 
use of volunteers in biodiversity (e.g., floral or faunal) 
surveys has becoming popular in recent years, yet a 
number of studies (e.g., Rinvall et ah, 2000; Lovell et 
ah, 2009; Sewell et al., 2010) showed that 
volunteer-based biodiversity surveys could produce data 
significantly different from that traditionally collected 
by professionals. These studies also suggested ways, 
such as using two volunteers to replace one 
professional, to reduce these differences. These studies 
emphasized the importance of not only suitable 
methodology, but also the training of personnel 
involved in biodiversity surveys. Finally, we hope this 
study would provide an anchor for pushing for 
meaningful legislation and training programs that would 
support significant improvements in EIAs. 
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